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ABSTRACT 

Effective  Command  and  Control  (C2)  is  enabled  by  infonnation  technology,  but  the  potential  of 
C2  through  virtual  environments  offers  opportunities  for  radical  increases  in  awareness  and 
performance.  To  realize  such  radical  increases,  we  are  engaged  in  a  campaign  of  experimentation 
to  assess  C2  in  virtual  environments  through  the  ELICIT  (Experimental  Laboratory  for 
Investigating  Collaboration,  Information-sharing,  and  Trust)  multiplayer  online  counterterrorism 
intelligence  game.  Currently  ELICIT  has  only  a  primitive  textual  interface.  This  research  seeks 
to  investigate  the  impact  of  translating  game-play  into  more  immersive  virtual  environments, 
where  participants  interact  as  avatars  and  in  modes  that  are  reminiscent  of  physical  experiences. 
Indeed,  drawing  from  research  in  modeling  and  virtual  environments,  we  hypothesize  that  a 
more  immersive  virtual  environment  offers  potential  to  improve  perfonnance.  Building  on 
previous  work  in  establishing  design  parameters  for  virtual  environments,  this  research  moves 
toward  a  more  complete  integration  of  ELICIT  into  the  virtual  space.  This  is  accomplished  by 
augmenting  basic  game-play  functionality  with  more  complex  interactions  in  the  virtual  space 
and  by  investigating  a  fully  automated  agent-based  experimentation  utilizing  ELICIT  in  virtual 
environments.  The  research  described  in  this  paper  explores  the  impact  of  virtual  environments 
on  game-play  and  seeks  to  understand  which  aspects  of  C2  can  be  perfonned  better  through 
virtual  environments  than  their  physical  counterparts. 
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INTRODUCTION 


Modern  military  organizations  have  adapted  and  evolved  over  many  centuries  and 
millennia,  respectively.  Hierarchical  C2  organizations  in  particular  have  been  refined 
longitudinally  (e.g.,  through  iterative  combat,  training  and  doctrinal  development)  to  become 
very  reliable  and  effective  at  the  missions  they  were  designed  to  accomplish.  However,  recent 
research  suggests  that  the  Hierarchy  may  not  represent  the  best  organizational  approach  to  C2  in 
all  circumstances  (Nissen,  2005),  particularly  where  the  environment  is  unfamiliar  or  dynamic. 
Indeed,  alternate,  more  flexible  C2  organizational  approaches  such  as  the  Edge  have  been 
proposed  (Alberts  &  Hayes,  2003)  to  overcome  Hierarchy  limitations,  but  the  same  recent 
research  suggests  that  the  Edge  may  not  represent  the  best  organizational  approach  to  C2  in  all 
circumstances  either,  particularly  where  the  environment  is  familiar  and  stable. 

Of  course,  the  Hierarchy  and  Edge  both  represent  organizational  archetypes  (Orr  & 
Nissen,  2006),  each  of  which  offers  considerable  latitude  in  terms  of  detailed  organizational 
design  and  customization.  For  instance,  recent  research  demonstrates  further  how  the 
perfonnance  of  both  Hierarchy  and  Edge  organizations  is  sensitive  to  factors  such  as  network 
infrastructure,  professional  competency  and  other  factors  that  can  be  affected  through  leadership, 
management  and  investment  (Gateau,  Leweling,  Looney,  &  Nissen,  2007).  With  incessant 
advances  in  information  technology  (IT)  that  appear  to  be  continuing,  one  may  be  able  to 
overcome  the  limitations  inherent  in  Hierarchy,  Edge  or  other  organizations  or  even  enable  such 
organizations  to  adapt — through  IT — to  shifting  conditions. 

This  notion  is  fundamental  to  Network  Centric  Operations  (NCO),  where  people  and 
organizations  operate  principally  in  network-enabled  virtual  environments  as  opposed  to  their 
physical  counterparts.  Unfortunately,  empirical  evidence  to  support  the  asserted  superiority  of 
NCO  remains  sparse,  and  the  capability  enhancing  properties  of  virtual  environments  remain 
more  in  the  domain  of  lore  than  empirical  assessment.  To  remedy  such  empirical  sparseness,  we 
continue  a  campaign  of  laboratory  experimentation  using  the  ELICIT  (Experimental  Laboratory 
for  Investigating  Collaboration,  Infonnation-sharing,  and  Trust)  multiplayer  online 
counterterrorism  intelligence  game  to  understand  the  comparative  advantages  and  disadvantages 
of  alternate  C2  approaches,  organizational  forms,  technologies  and  like  aspects  across  a  range  of 
current  and  anticipated  operational  environments. 

In  this  particular  stream  of  research  we  focus  specifically  on  the  information  technology 
(IT)  that  enables  C2  through  virtual  environments,  looking  intensively  for  radical  increases  in 
awareness  and  perfonnance  through  immersion  in  such  environments.  Indeed,  drawing  from 
substantial  research  in  modeling  and  virtual  environments,  we  argue  that  a  more  immersive 
virtual  environment  offers  potential  to  improve  performance.  Currently  ELICIT  has  only  a 
primitive  textual  interface.  This  research  seeks  to  investigate  the  impact  of  translating  game-play 
into  more  immersive  virtual  environments,  where  participants  interact  as  avatars  and  in  modes 
that  are  reminiscent  of  physical  experiences. 

Building  on  previous  work  in  establishing  design  parameters  for  virtual  environments, 
this  research  moves  toward  a  more  complete  integration  of  ELICIT  into  the  virtual  space.  This  is 
accomplished  by  augmenting  basic  game-play  functionality  with  more  complex  interactions  in 
the  virtual  space  and  by  investigating  a  fully  automated  agent-based  experimentation  utilizing 
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ELICIT  in  virtual  environments.  The  research  described  in  this  paper  analyzes  the  impact  of 
virtual  environments  on  game-play  and  seeks  to  understand  what  aspects  of  C2  can  be  perfonned 
better  through  virtual  environments  than  their  physical  counterparts. 

BACKGROUND 

The  aim  of  this  research  is  to  study  the  impact  of  ELICIT  game-play  with  participants 
who  are  immersed  in  a  3D  virtual  environment.  By  definition,  these  environments,  where  players 
interact  as  avatars,  are  open-ended,  with  the  spectrum  of  visual  and  auditory  stimuli  available. 
The  first  step  in  approaching  this  project  is  to  determine  both  which  virtual  world  product  would 
be  leveraged  for  experimentation,  and  what  type  of  environment  would  be  created  to  house  those 
experiences. 

To  that  end,  the  available,  suitable  virtual  worlds  were  evaluated  based  on  the  following  criteria: 

1)  Accessibility:  Virtual  environment  platform  must  be  universally  accessible  between 
operating  systems.  The  environment  must  be  intuitive  in  tenns  of  its  usage,  reducing  to 
the  bare  minimum  the  training  load  for  all  participants. 

2)  Communication:  Virtual  world  platfonn  must  allow  for  both  text  and  voice 
communication.  This  communication  apparatus  must  be  straightforward  to  learn  and 
simple  to  use. 

3)  ELICIT  specific  requirements:  ELICIT  game-play  is  a  proctored  clue-based  game  that 
collects  information  about  game-play  in  a  database  so  that  researchers  may  examine 
statistical  records  of  each  game  round.  Currently  the  game  functions  are  handled  by 
JavaScript,  so  ideally,  the  virtual  environment  will  support  the  inclusion  of  JavaScript 
without  significant  re-coding  of  the  environment. 

4)  Customization:  The  virtual  environment  must  support  complete  customization  of  spatial 
aspects.  This  customization  can  preferably  be  done  in  an  expedient  manner.  (Hudson  & 
Nissen,  2010) 

While  many  of  the  available  virtual  world  products  could  be  tailored  to  the  ELICIT 
experiments,  the  public  world  of  Second  Life  was  chosen  as  the  most  readily  adaptable  to  this 
purpose,  based  on  cost,  speed  of  development,  and  native  tools  to  enable  ELICIT  game-play, 
without  extensive  programming. 

Having  determined  the  platfonn  for  experimentation,  the  next  step  in  preparing  the 
environment  was  to  design  and  develop  a  suitable  neutrally  toned  space,  so  that  environmental 
elements  did  not  overtly  stimulate  participants.  This  allows  researchers  to  examine  the  core 
interactions  between  participants,  while  avoiding  cognitive  overload  due  to  excessive  and 
extraneous  elements.  This  component  of  the  experiential  design  is  critical  for  adjacent  proposed 
research  utilizing  Lunctional  Magnetic  Resonance  Imaging  (fMRI)  to  evaluate  neurological 
activity  during  ELICIT  game-play;  the  more  neutral  the  base  environment  is,  the  more  accurate 
the  fMRI  results  will  be  in  mapping  brain  activity  during  game-play  in  the  virtual  environment. 

To  ensure  a  neutral  experience,  researchers  developed  a  rubric  of  suitable  textures,  layouts, 
and  color  combinations  that  would  render  the  space  as  neutral  as  possible.  Not  only  does  this 
provide  a  baseline  experience  for  the  experiments,  but  also  makes  available  the  possibility  of 
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adding  elements  to  stimulate  emotional  states  conducive  to  game-play  at  a  later  time,  and  then  to 
compare  the  ELICIT  results  from  the  various  trials. 

The  ELICIT  game-play  itself  is  nested  into  the  virtual  environment  by  loading  the  webpage, 
and  playing  the  game  in  the  same  manner  that  it  would  be  played  in  other  contexts.  Desks  with 
computer  monitors  were  created  within  the  virtual  space,  and  participants  interacted  with  the 
ELICIT  game  through  and  within  the  virtual  environment  in  a  way  that  is  analogous  with  the 
traditional  web-based  version  of  the  game.  The  difference  being  that  the  participants  are  co¬ 
located  in  a  virtual  environment  while  they  play  ELICIT. 

CANVIRTUAL  E  NV I  R  O  NM  E  NT  S  I  M  PR  OV  E  PERFORMANCE? 

The  overarching  aim  of  this  research  is  to  detennine  whether  or  not  virtual  environments 
can  improve  perfonnance  in  a  C2  context.  Research  suggests  that  these  environments  are  highly 
suited  to  increase  performance  in  training  and  learning  situations.  However,  can  and  will  this 
substantiated  application  of  virtual  environments  translate  into  success  with  command  and 
control  situations? 

One  primary  example  of  the  application  of  virtual  environments  that  increased 
perfonnance  in  a  training  context  is  the  Canadian  border  simulation  run  out  of  Loyalist  College. 
In  this  experience,  students  in  the  Customs  and  Immigration  program  participated  in  simulated 
border  crossing  interview  as  a  method  of  rehearsing  mandatory  questioning  skills.  The  simulated 
crossing,  also  developed  in  Second  Life,  accurately  recreates  a  border  checkpoint,  and  each 
student  in  the  class  is  given  a  chance  to  enter  the  booth  and  conduct  interviews  with  travelers 
entering  Canada. 

The  student  border  officers  adhere  to  strict  interview  guidelines  and  must  conduct  the 
questioning  in  a  professional  and  representative  manner.  Each  student  avatar  is  provided  with 
basic  training  in  operating  the  virtual  environment,  and  each  has  a  border  officer  uniform  for 
their  avatar.  The  students  stand  in  a  border  control  booth,  and  have  access  to  primary  relevant 
traveler  information  (passport,  license  plate)  displayed  on  the  monitor  within  the  booth  (Ligure 
1).  The  students  interview  the  traveler,  ensuring  that  the  infonnation  provided  verbally  synchs 
with  the  database  of  information  relative  to  that  traveler.  They  then  decide  either  to  admit  the 
traveler  into  Canada,  or  refer  him  to  secondary  or  immigration  screening.  Each  traveler  interview 
is  followed  up  with  an  after-action  review,  where  questions  and  comments  related  to  the 
interview  could  be  discussed. 
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Figure  1:  Loyalist  College  Border  Simulation 

In  contrast  to  previous  uses  of  in-class  role-play  to  teach  these  interview  skills,  the 
Loyalist  College  students  significantly  out-perfonned  earlier  classes  by  leveraging  the  virtual 
environment.  Grades  associated  with  these  interview  skills  increased  28%  in  the  first  trial,  and  an 
additional  9%  with  a  subsequent  group  of  students.  Additionally,  students  reported  an  increase  in 
confidence,  and  reported  a  depth  of  knowledge,  not  associated  with  previous  in-class  experiences 
(deGast-Kennedy  &  Hudson,  2009). 


■  Interview  58%  86%  94% 

Skills 


■  Interview  Skills 


Figure  2:Increased  success  using  virtual  environment 
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The  success  of  the  Loyalist  College  experience  led  to  collaboration  with  the  Canada 
Border  Services  Agency  (CBSA),  who  agreed  to  trial  this  simulated  training  environment  with 
new  recruits  to  the  agency.  Applied  to  a  distributed  pre-training  course  for  new  recruits,  the 
intent  of  the  simulation  in  this  context  is  to  prepare  learners  for  arrival  at  basic  training  by  giving 
them  a  sense  of  the  complexities  the  role  would  require.  The  CBSA  results  echo  those  from 
Loyalist  College,  seeing  recruits  who  participated  in  the  virtual  environment  simulation  being 
39%  more  successful  at  the  first  testing  milestone  than  those  who  did  not  participate  in  the 
virtual  world  experience  (Hudson  &  Nowosielski,  2009). 

ATTRIBUTES  OF  V I  R  T  U  AL  E  NV I  R  ONM  E  NT  S  L  E  ADI  NG  TO  B  ETT  E  R 
PERFORMANCE 

The  border  simulation  at  Loyalist  College  is  a  hallmark  of  successful  application  of 
virtual  environments  for  applied  learning  experiences.  But  what  are  the  attributes  of  virtual 
environments  that  led  to  this  success,  and  can  they  be  harnessed  and  applied  to  non-instructional 
situations,  such  as  the  ELICIT  counter-terrorism  game?  There  are  several  factors  in  the  design  of 
these  experiences  that  lend  themselves  to  the  development  of  an  appropriate  learning  experience 
in  virtual  environments. 

In  Learning  in  3D,  Karl  Kapp  and  Tony  O’Driscoll  (2010)  identify  aspects  of 
successfully  designed  learning  experience  that  leverage  the  inherent  attributes  of  virtual 
environments.  Those  elements  include,  participant  centered  experience,  contextually  situated 
learning,  and  experiences  that  fully  utilize  the  style  of  interactions  and  experiences  native  to 
virtual  environments,  including  the  identification  of  self  with  avatar  within  the  environment 
(presence),  and  the  sense  of  sharing  that  environment  with  others  (co-presence). 

Analyzing  the  border  simulation  in  this  light,  we  see  that  it  capitalizes  on  all  of  these 
native  attributes  as  a  pathway  to  success.  It  is  contextually  relevant,  having  participants  interact 
within  a  virtual  border  crossing  environment;  the  participants  themselves  drive  the  action  or 
activity,  making  this  experience  participant  centered;  the  participants  assume  the  role  they  are 
enacting  by  dressing  their  avatars  in  the  appropriate  uniform  for  the  experience;  and  the 
experience  is  dependent  upon  meaningful  interactions  that  require  both  border  officer  and 
traveler  to  be  mutually  present  within  the  simulation. 

The  promise  of  increased  performance  in  virtual  environments  is  connected  to  those 
attributes  native  to  these  environments  that  make  them  distinct  from  other  mediated  experiences 
(web,  teleconference,  and  so  forth).  In  order  to  realize  enhanced  perfonnance,  virtual  experiences 
must  align  themselves  with  those  features  in  order  to  capitalize  on  these  unique  attributes. 

ELICIT  IN  VIRTUAL  ENVIRONMENTS 

The  initial  phase  of  experimentation  playing  ELICIT  within  the  virtual  environment 
failed  to  meet  the  expectation  that  this  platfonn  would  enhance  performance.  Two  aspects  of  the 
ELICIT  game-play  experiment  conducted  in  the  virtual  environment  were  compared  to  trials  run 
in  the  physical  environment.  These  two  aspects,  time  and  accuracy,  are  critical  to  ELICIT,  and, 
of  course,  crucial  for  effective  command  and  control  functions. 
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The  results  demonstrate  that  while  there  was  no  measurable  differences  in  the  time  it  took 
both  groups  to  play  ELICIT,  the  group  that  was  playing  from  within  the  virtual  environment  was 
less  successful  in  terms  of  accuracy  than  the  corresponding  group  playing  in  a  physical 
environment  (Bergin,  Adams,  Andraus,  Hudgens,  Lee  &  Nissen,  2010),  see  Table  1.  In  order  to 
understand  these  results,  we  will  compare  the  experimental  constructs  for  each,  and  then  evaluate 
the  virtual  experience  in  light  of  the  inherent  strengths  and  unique  qualities  of  the  virtual 
environment. 


Table  1:  Table  of  results  from  initial  ELICIT  experiments  in  virtual  environments 
comparing  computer  mediated  (CM)  with  face  to  face  (LTL)  results. 


Variable 

Mean 

N 

Standard 

Deviation 

Standard  Error 

ID  Time  CM  (in  seconds) 

2685 

14 

219 

58 

ID  Time  LTL  (in  seconds) 

2554 

14 

279 

74 

Who  Score  CM 

.140 

14 

.363 

.097 

Who  Score  LTL 

.790 

14 

.426 

.114 

What  Score  CM 

.321 

14 

.249 

.066 

What  Score  LTL 

.536 

14 

.365 

.098 

Where  Score  CM 

.570 

14 

.514 

.137 

Where  Score  LTL 

.790 

14 

.426 

.114 

When  Score  CM 

.262 

14 

.297 

.079 

When  Score  LTL 

.333 

14 

.320 

.086 

Identify  Composite  CM 

.324 

14 

.206 

.055 

Identify  Composite  LTL 

.610 

14 

.327 

.087 

On  the  surface,  the  two  groups,  one  virtual  and  one  physical  were  playing  the  same 
ELICIT  game.  Both  were  interfacing  with  the  ELICIT  game  through  a  web  browser,  one  group 
in  a  physical  room  with  computer  monitors,  and  the  other  within  a  virtual  space  designed  to 
mimic  the  physical  room,  also  with  computer  monitors  stationed  at  desks.  However,  in 
analyzing  the  mode  of  interactions  with  ELICIT,  the  physical  space  afforded  the  comforts  of  the 
familiar,  allowing  participants  to  interact  with  ELICIT  in  what  can  only  be  called  a  “normal”  or 
“regular”  manner  compared  to  the  way  they  would  interact  with  any  web  or  computer-based 
application. 
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On  the  other  hand,  the  virtual  environment  players,  in  addition  to  negotiating  the 
unfamiliar  virtual  space,  played  the  ELICIT  game  in  a  manner  that  eclipsed  the  potential  benefits 
of  the  virtual  environment.  The  virtual  computer  interface  requires  users  to  interact  in  an  extreme 
close-up  view,  excluding  all  other  visual  information  from  their  sight.  Therefore,  the  virtual 
subjects  were  not  necessarily  experiencing  themselves  within  the  virtual  space 
(identification/presence),  nor  were  they  experiencing  the  other  participants  who  were  co-located 
in  the  same  space  (co-presence),  in  the  way  their  physical  counterparts  were,  who  were  clearly 
part  of  a  team  of  participants.  Instead  their  complete  focus  was  dominated  by  the  computer 
screen  and  the  ELICIT  interface  (see  Figure  3).  Ironically,  it  could  be  observed,  that  they  were 
less  immersed  in  the  virtual  environment  (the  aim  of  the  experiment)  and,  in  fact,  more 
immersed  in  the  text-base  ELICIT  web  interface. 


Figure  3:  VE  ELICIT  interface  obscures  view  of  environment  in  initial  experiments 

Furthermore,  building  on  the  known  advantages  for  interaction  within  virtual 
environments,  the  participants  in  this  experiment  were  in  a  neutral  space,  which  may  or  may  not 
have  registered  as  contextually  relevant  to  them.  Conversely,  the  physical  participants  were  in  a 
boardroom  setting,  one  that  evokes  a  seriousness  of  task,  allowing  for  one  basic  reading  of  the 
semiotics  of  this  kind  of  environment. 

Building  out  from  the  best  practices  for  applying  virtual  environments,  one  could  argue 
that,  in  this  case,  performance  was  not  enhanced  because  the  experiments  failed  to  engage 
participants  with  the  inherent  strengths  of  virtual  environments.  Also,  the  interface  for  the 
ELICIT  game-play  as  developed  in  the  virtual  environment  did  not  in  any  way  further  immerse 
the  participants  in  the  game.  And,  because  there  was  no  interaction  between  participants  during 
the  experiment,  the  co-presence  advantage  of  virtual  environments  was  rendered  moot. 
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While  the  results  of  the  initial  experimentation  did  not  further  the  hypothesis  of  the 
advantages  of  virtual  environments,  the  relative  failure  of  this  first  trial  points  to  the  need  to 
further  integrate  the  ELICIT  game  within  the  virtual  space  so  as  to  allow  the  inherent  strengths 
of  these  environments  to  register  within  the  experiment.  Once  there  is  sufficient  integration  of 
ELICIT,  there  still  exists  a  belief  that  perfonnance  enhancements  will  be  evident. 


ELICIT  INTEGRATION 

The  initial  phase  of  this  research  project  was  to  enable  experimentation  with  ELICIT  to 
be  located  within  a  3D  virtual  environment.  The  next  steps  involve  the  further  integration  of 
ELICIT  within  the  virtual  environment  by  providing  facilitators  and  researchers  with  the  ability 
to  play  the  game  and  conduct  experiments  in  a  way  that  closely  evokes  the  physical  experience. 

The  challenge  in  extending  functionality  beyond  the  web-based  ELICIT  game  is  that  each 
virtual  environment  has  pre-detennined  types  of  interactions  that  are  hard-wired  in  their 
platforms.  For  example,  Second  Life,  while  allowing  input  onto  a  webpage  (like  the  ELICIT  web 
game),  does  not  allow  for  simple  Player  to  Player  (P2P)  interactions,  like  handing  a  physical 
clue.  Those  types  of  P2P  interactions  are  removed  from  the  manner  in  which  the  same 
experience  would  be  handled  in  a  physical  setting. 

For  example,  the  simple  task  of  passing  a  physical  clue  from  one  participant  to  another, 
or  posting  that  clue  on  a  common  whiteboard,  is  severely  limited  in  Second  Life.  If  the  game- 
play  is  being  conducted  through  the  ELICIT  website,  there  is  no  content  that  is  technically  “in” 
Second  Life  to  be  manipulated;  the  content  continues  to  rest  solely  on  the  ELICIT  website. 

The  only  way  to  bypass  this  restriction  is  to  host  the  ELICIT  content  within  the  virtual 
environment  itself.  This  is  the  case  with  a  “paper  clue”  version  of  ELICIT  that  is  being 
developed  for  Second  Life.  Clearly  there  is  no  paper  in  the  virtual  world,  but  also,  there  is  no 
ready  and  simple  manner  to  create  a  virtual  piece  of  paper,  and  then  distribute  or  present  those 
clues  to  participants.  Rather,  a  complex  and  removed  system  of  interaction  needs  to  be  developed 
to  handle  even  the  most  elementary  infonnation  transaction  within  the  virtual  space. 

Therefore  developers  must  seek  technological  analogues  in  the  virtual  space  to  physical 
interactions.  One  approach  to  replacing  tactile  interactions  is  through  the  use  of  a  Head’s  Up 
Display  (HUD).  The  HUD  is  a  programmed  display  that  loads  on  the  screen  in  from  of  the  player 
in  the  virtual  environment  (Figure  4).  In  this  case,  the  HUD  would  display  any  clues  that  were 
passed  to  the  individual  player,  and  would  allow  the  player  to  further  share  or  post  that  clue  in  a 
manner  that  resembles  the  physical  manipulation  of  clues  in  a  paper  version  of  the  game. 
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Figure  4:  ELICIT  "paper  clue"  HUD  prototype 

This  style  of  interaction  raises  many  questions  regarding  the  influence  of  the  interface  in 
such  a  transaction  versus  the  lack  of  interface  in  a  physical  version  of  the  game.  Can  these  two 
styles  of  “paper  clue”  ELICIT  even  be  compared  as  somewhat  equal  versions  of  the  game? 
Clearly,  the  HUD  will  enable  players  to  functionally  replicate  physical  game  play,  even  though 
the  medium  (digital  versus  paper)  has  shifted.  It  is  believed  despite  the  dramatic  differences  in 
format,  that  the  essence  of  the  game-play  will  remain  integral  in  such  an  approach. 

However,  there  is  not  the  same  confidence  that  a  hybrid  model  of  game  play  could  be 
sustained  within  the  virtual  world,  where  participants  could  at  once  interface  with  the  main 
ELICIT  game  via  the  website,  and,  as  well,  manipulate  the  content  of  the  clues  in  an  interactive 
and  interpersonal  manner.  One  of  the  key  limitations  of  a  public  or  “off  the  shelf’  virtual  world, 
is  the  lack  of  programming  support  to  insert  content  into  the  environment.  The  HUD  could  also 
be  employed,  however,  again  without  the  content  being  hosted  within  the  virtual  environment, 
there  are  numerous  hurdles  to  overcome  in  communicating  with  the  base  game  (hosted  on  the 
internet)  to  allow  for  content  interactions  within  the  virtual  world. 

One  approach  to  compensate  for  the  limitations  of  interacting  with  ELICIT  in  the  virtual 
environment  is  to  transfer  game-play  from  a  computer  monitor  situated  in  the  space  to  a  HUD 
interface.  It  is  believed  that  this  will  correct  the  limited  viewpoint  created  by  the  computer 
monitor,  which  had  the  unintended  effect  of  blocking  a  view  of  the  other  participants.  This 
approach  will  enable  participants  to  both  interact  with  the  ELICIT  client,  and  to  interact  with  one 
another  and  their  environment  (Figure  5).  This  could  provide  a  solution  to  the  static  virtual 
workstations  of  the  initial  experiment  that,  as  we  have  shown,  did  not  support  enhanced 
performance.  Whether  or  not  this  change,  slight  as  it  is,  would  demonstrate  any  significant 
changes  in  results,  is  uncertain.  However,  given  that  it  would  align  the  experience  with  best- 
practices  guidelines  for  success  in  virtual  environment,  it  is  worthy  of  investigation,  and  leads  to 
new  hypotheses  for  further  experimentation  of  ELICIT  within  virtual  environments. 
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Hypothesis  1:  Player's  performance  of  the  ELICIT  task  game  will  increase  within  the  virtual 
environment  when  the  native  strengths  of  those  environments  are  leveraged. 

Hypothesis  2:  Player's  performance  of  the  ELICIT  task  game  will  increase  within  the  virtual 
envi ronment  when  the  interface  for  game-play  are  aligned  with  their  physical  analogues. 


Figure  5:  Full  ELICIT  game-play  and  whiteboard  functionality  HUD  prototype 

AGENT-BASED  ELICIT 

The  virtual  worlds  industry  and  available  products  and  platforms  have  expanded  greatly 
in  the  past  two  years.  To  say  that  these  products  change  quickly  is  a  profound  understatement. 
From  the  inception  of  this  research,  the  evolution  of  virtual  environments  has  been  staggering. 
Little  more  than  one  year  ago,  some  of  the  functionality  that  is  required  for  ELICIT  game-play 
was  not  available  within  Second  Life.  The  preliminary  research  to  determine  which  virtual  world 
platform  to  use  indicated  that  there  are  a  range  of  environments  that  would  support  ELICIT,  but 
only  as  a  native  “within  the  browser”  application  that  is  played  in  the  virtual  world. 

The  agent -based  version  of  ELICIT  (abELICIT)  allows  for  a  single  player  to  participate 
in  the  ELICIT  game,  while  the  other  players  are  controlled  by  autonomous  or  programmed 
agents.  The  aim  of  integrating  agent-based  game-play  within  the  virtual  world  is  the  subject  of 
this  second  phase  of  development  and  experimentation.  There  are  multiple  reasons  why  such  a 
state  would  be  advantageous,  primarily  the  ability  to  conduct  ELICIT  trials  with  a  single  or 
limited  number  of  participants. 

While  the  Second  Life  platform  is  reliable  for  developing  and  launching  a  rapid  prototype 
of  ELICIT  in  virtual  worlds,  it  is  wholly  unfriendly  as  regards  large  programmed  elements  added 
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on  to  the  world.  The  complex  backend  of  ELICIT,  which  not  only  serves  the  experience  for 
participants,  but  also  tracks  the  manner  and  choices  made  within  the  game,  is  by  definition 
incompatible  with  a  virtual  world.  This  is  true  of  straightforward  informatics,  as  shown  above, 
and  even  more  so  with  driving  “unmanned”  avatars  within  a  virtual  space. 

As  is  the  case  in  the  entire  virtual  environment  sector  of  technology,  we  see  solutions 
arriving  just  in  time  to  support  the  evolution  of  needs  and  applications  for  these  spaces.  So  while 
there  is  a  relative  impossibility  in  adapting  agent-based  ELICIT  game-play  into  a  public  world 
like  Second  Life,  there  are  emergent  rigorous  solutions  that  provide  a  range  of  customization  that 
could  support  such  a  development. 

One  leading  solution  is  the  Unity3D  authoring  toolset.  While  not  a  virtual  world,  per  se, 
Unity  offers  developers  the  unprecedented  ability  to  design  and  program  custom  virtual  world 
solutions  in  a  streamlined  and  affordable  manner.  This  product  is  very  much  like  videogame 
authoring  tools,  with  the  important  exception  that  it  supports  MMO  +  VW  (massive  multi-player 
online  +  virtual  world)  style  interactions.  As  well,  the  Unity  tools  create  virtual  worlds  that  can 
be  accessed  through  any  web  browser,  without  the  need  for  expensive  hardware  upgrades. 

We  know  that  the  ability  to  construct  a  game  environment  has  been  available  for  some 
time.  However,  a  game  is  not  a  virtual  world,  in  that  virtual  worlds  allow  for  a  range  of  choice 
and  interactions  usually  not  supported  in  traditional  videogames.  In  a  videogame,  you  play  the 
game,  but  you  can’t  necessarily  communicate  or  inter-relate  with  the  other  participants  in  a  free 
flowing  manner.  In  most  cases,  these  interactions  are  scripted  in  the  fabric  of  the  game  to  keep 
the  interactions  relative  to  the  game  narrative. 

However,  in  a  virtual  world,  while  one  may  play  a  game  (like  ELICIT  in  Second  Life), 
there  exists  the  ability  to  interact  with  other  participants,  to  adapt  or  change  the  environment  to 
suit  the  experience,  or  to  have  multiple  environmental  constructs  to  support  a  range  of 
experimentation.  So  while  not  as  “out  of  the  box”  as  a  Second  Life,  these  virtual  world  authoring 
tools  allow  for  integration  of  complex  programmed  elements  while  maintaining  the  flexibility 
native  to  virtual  worlds. 

These  new  authoring  tools  are  powerful  enough  to  accomplish  complex  hybrid  designs 
for  experiments.  So  while  an  agent-based  ELICIT  in  a  virtual  world  like  Second  Life  would  be 
impossible  to  achieve,  it  is  highly  feasible  using  the  Unity3D  toolset.  So  that  not  only  could  a 
custom  environment  be  created  but  also  that  environment  could  support  autonomous  agents 
playing  ELICIT  as  avatars,  and  interacting  with  the  live  participants  in  a  very  realistic  manner. 
Additionally,  this  type  of  experience  would  also  be  able  to  support  all  of  the  data  collection 
methodologies  that  are  inherent  in  web  based  ELICIT  as  part  of  the  program  code  in  the  virtual 
world,  without  having  to  bridge  the  two  applications  to  have  a  fully  functioning  virtual  ELICIT. 

While  these  developments  render  it  possible  to  achieve  this  level  of  integration  within  the 
virtual  environment,  further  research  is  required  to  determine  the  process  by  which  such 
integration  will  be  achieved. 
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CONCLUSION 


Virtual  environments  offer  the  potential  for  increases  in  awareness  and  perfonnance  for 
C2.  To  realize  such  gains,  we  are  engaged  in  a  campaign  of  experimentation  to  assess  C2  in 
virtual  environments  through  the  ELICIT  multiplayer  online  counterterrorism  intelligence  game. 
Currently  ELICIT  has  only  a  primitive  textual  interface.  This  research  seeks  to  investigate  the 
impact  of  translating  game-play  into  more  immersive  virtual  environments.  However,  a  simple 
transposition  of  real  world  experimental  environments  into  virtual  spaces  did  not  yield  any 
improvement  in  task  perfonnance.  In  analyzing  the  experimental  construct,  it  is  observed  that  the 
replication  of  the  physical  experimental  space  diminished  the  participant’s  experience  of  the 
strengths  of  virtual  environments  for  collaboration  and  perfonnance.  Those  strengths,  the 
identification  of  the  self  with  the  avatar  (presence),  and  the  sense  of  sharing  the  environment 
with  others  (co-presence)  have  been  identified  as  differentiators  that  render  virtual  environments 
unique  in  relation  to  other  infonnation  technologies.  To  further  this  research,  we  intend  to  recast 
the  environmental  design  to  dovetail  with  these  native  attributes.  It  is  hypothesized  that  in  doing 
so,  ELICIT  game-play  within  virtual  environments  will  be  augmented,  and  that  participants  will 
demonstrate  increases  in  their  game-play  performance. 


Acknowledgements:  The  research  described  in  this  article  was  funded  in  part  by  the  Command  and 
Control  Research  Program  through  the  Center  for  Edge  Power  at  the  Naval  Postgraduate  School. 


13 


REFERENCES 


Alberts,  D.  S.,  &  Hayes,  R.  E.  (2003).  Power  to  the  edge:  Command  and  control  in  the 
information  age.  Washington,  DC:  Command  and  Control  Research  Program. 

Bergin,  R.  D.,  Adams,  A.  A.,  Andraus,  R.,  Hudgens,  B.  J.,  Lee,  J.  G.,  &  Nissen,  M.  E.  (2010). 
Command  and  control  in  virtual  environments:  Laboratory  experimentation  to  compare 
virtual  with  physical.  Santa  Monica,  CA. 

deGast-Kennedy,  K  &  Hudson,  K.  (2009).  Canadian  border  simulation  at  Loyalist  College. 
Journal  of  Virtual  Worlds  Research,  2(1) 

Gateau,  J.  B.,  Leweling,  T.  A.,  Looney,  J.  P.,  &  Nissen,  M.  E.  (2007).  Hypothesis  testing  of  edge 
organizations:  Modeling  the  C2  organization  design  space.  Proceedings  International 
Command  &  Control  Research  &  Technology  Symposium,  Newport,  RI. 

Hudson,  K.  &  Nissen,  M.  E.  (2010).  Command  and  control  in  virtual  environments:  Designing  a 
virtual  environment  for  experimentation.  Santa  Monica,  CA. 

Hudson,  K.  &  Nowosielski,  L.  (2009,  April).  Canadian  border  simulation.  Poster  session 

presented  at  the  Federal  Consortium  on  Virtual  Worlds  Conference,  National  Defense 
University,  Washington,  DC. 

Kapp,  Karl  M.  &  O’Driscoll,  Tony.  (2010).  Learning  in  3D:  Adding  a  new  dimension  to 
enterprise  learning  and  collaboration.  San  Francisco,  CA:  John  Wiley  and  Sons. 

Nissen,  M.  E.  (2005).  Hypothesis  testing  of  edge  organizations:  Specifying  computational  C 
models  for  experimentation.  Proceedings  International  Command  &  Control  Research 
Symposium,  McLean,  VA. 

Orr,  R.  J.,  &  Nissen,  M.  E.  (2006).  Computational  experimentation  on  C2  models.  Proceedings 
International  Command  and  Control  Research  and  Technology  Symposium,  Cambridge, 
UK. 


14 


CENTER  FOR 

EXE 


POWER 


Understanding  the  potential  of 

virtual  environments 

for  improving  C2  performance 


Ken  Hudson,  Loyalist  College 
Mark  E.  Nissen,  US  Naval  Postgraduate  School 


16th  International  Command  &  Control  Research  &  Technology  Symposium 


/ CENTER  FOR 

>Cedce 

POWER 


•  Background 

•  Can  VE  improve  performance? 

•  Initial  experiment  and  analysis 

•  Aspects  of  VE  leading  to  improved  performance 

•  Current  experimental  approach 

•  Next  steps 


Outline 


CENTER  FOR 


\EDCE 

POWER 


Background 


•  ELICIT:  multiplayer  online  game 

—Counterterrorism  intelligence  domain 
—Like  game  Clue :  ID  fictitious  terrorist  plots 

•  Used  for  C2  experimentation 

—Examine  org  forms,  learning,  trust,  culture, 
others 

—Very  thin,  textual  interface 

•  Interested  in  extending  into  immersive  VE 


CENTER  FOR 


LEDCE 

POWER 


•  Customizable  platform  for  interaction 

•  Synchronous  and  persistent  network 

•  People  are  represented  as  avatars 

•  Flexibility  of  usage 


CENTER  FOR 


LEOCE 

POWER 


a'/ilita  Varrial 


Gtones  Haldeniian 


ead  yaa/j  Co  uni 

!t!)®  fe  rlyfi/illf/C 


C!aire!!e  jVloufJJn 


jliilffi'/Jet  Fosse!! 


GaUiffi 

VidJ 


CENTER  FOR 


HEDCE 

POWER 


Design  elements  to  support 
Neutral  Valence 


•  Interior  space  only  free  from  natural  world 
complexities 

•  Subtle  texturing  of  spaces 

•  Constant  ambient  lighting 

•  Narrow  range  of  environmental  stimuli 


CENTER  FOR 


HEDCE 


POWER 


Neutral  space  prototype 


CENTER  FOR 


HEDCE 

POWER 


CENTER  FOR 


LEOCE 

POWER 


9 


CENTER  FOR 


LEOCE 

POWER 


100% 

80%  - 
60%  - 
40%  - 
20%  - 
0%  - 

2007 

■  Interview  58% 
Skills 


■  Interview  Skills 


10 


CENTER  FOR 

EDGE 


POWER 


11 


✓  center  for 

HEDCE 


POWER 


Hybrid  Room 


CENTER  FOR 


LEOCE 

POWER 


Me  Communic 

ate  World  Build  Help  Advanced 

L$  556  Buy  L$ 

8-21AMPDT  II  *)) 

<  >  # 

■  0  Loyalist  Exploration,  Loyalist  Exploration  -  General  Q 

L  Search  j 

Infinite  Spaces 

Loyalist  Exploration 

CENTER  FOR 


LEDCE 

POWER 


14 


CENTER  FOR 


HEDCE 

POWER 


Variable 

Mean 

N 

Standard 

Deviation 

Standard  Error 

ID  Time  CM  (in 

2685 

14 

219 

58 

seconds) 

2554 

14 

279 

74 

ID  Time  FTF  (in 
seconds) 

Who  Score  CM 

.140 

14 

.363 

.097 

Who  Score  FTF 

.790 

14 

.426 

.114 

What  Score  CM 

.321 

14 

.249 

.066 

What  Score  FTF 

.536 

14 

.365 

.098 

Where  Score  CM 

.570 

14 

.514 

.137 

Where  Score  FTF 

.790 

14 

.426 

.114 

When  Score  CM 

.262 

14 

.297 

.079 

When  Score  FTF 

.333 

14 

.320 

.086 

Identify  Composite  CM 

.324 

14 

.206 

.055 

Identify  Composite  FTF 

.610 

14 

.327 

.087 

15 


CENTER  FOR 


HEDCE 

POWER 


•  Participant  centered  experience 

•  Contextually  situated  learning 

•  Fully  utilize  VE  style  of  interactions 

•  Identification  of  self  with  avatar 

•  Sense  of  being  in  environment  (presence) 

•  Sense  of  sharing  environment  with  others 
(co-presence) 


16 


CENTER  FOR 


LEOCE 

POWER 


•  Seeks  to  harmonize  native  attributes  of 
virtual  environments  that  lead  to  enhanced 
performance  by  augmenting  user  interface 
and  allowing  for  a  broader  range  of 
interactions  than  the  initial  experiment. 


CENTER  FOR 


LEOCE 

POWER 


ELICIT  in  HUD 


A  Loyalist  Exploration.  Loyalist  Exploration  •  General  Q 


Lockdown  -  Boardro 


Me  Communicate  World  Build  Help  Advanced 


L$  2.036 


wNPG  Space- Ex  iS  Play  Area  UHN ICU  Nursing  Room  Recruitment  Loyalist  College  Par  Border  Simulation  Bruce's  Play  Area  Feeding  Edge/TCC  Digital  Ontano  Exploration  Layout 


<  >  #c. 


KMJCiTw*. 


S3 

TyjlPPlA.  wtifeboara  .-«i  Hour  L*»iHs 


Speak  •  *  Gesture  '  Move  View 


18 


CENTER  FOR 


LEOCE 

POWER 


19 


CENTER  FOR 


HEDCE 

POWER 


•  Proceed  with  experimentation  with  ELICIT 
game-play  through  HUD 

•  Enhanced  experimentation  using  'paper  clue' 
version  of  ELICIT  in  both  F2F  and  CM 

•  Explore  potentialities  of  integrated  abELICIT 
within  virtual  environment 


20 


